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ABSTRACT 


Introduction: In 2020, prior to COVID-19 vaccine rollout, the Brighton Collaboration created a priority list, 
endorsed by the World Health Organization, of potential adverse events relevant to COVID-19 vaccines. 
We adapted the Brighton Collaboration list to evaluate serious adverse events of special interest observed 
in MRNA COVID-19 vaccine trials. 
Methods: Secondary analysis of serious adverse events reported in the placebo-controlled, phase III ran- 
domized clinical trials of Pfizer and Moderna mRNA COVID-19 vaccines in adults (NCT04368728 and 
NCT04470427), focusing analysis on Brighton Collaboration adverse events of special interest. 
Results: Pfizer and Moderna mRNA COVID-19 vaccines were associated with an excess risk of serious 
adverse events of special interest of 10.1 and 15.1 per 10,000 vaccinated over placebo baselines of 
17.6 and 42.2 (95 % CI —0.4 to 20.6 and —3.6 to 33.8), respectively. Combined, the mRNA vaccines were 
associated with an excess risk of serious adverse events of special interest of 12.5 per 10,000 vaccinated 
(95 % CI 2.1 to 22.9); risk ratio 1.43 (95 % CI 1.07 to 1.92). The Pfizer trial exhibited a 36 % higher risk of 
serious adverse events in the vaccine group; risk difference 18.0 per 10,000 vaccinated (95 % CI 1.2 to 
34,9); risk ratio 1.36 (95 % CI 1.02 to 1.83). The Moderna trial exhibited a 6 % higher risk of serious adverse 
events in the vaccine group: risk difference 7.1 per 10,000 (95 % CI -23.2 to 37.4); risk ratio 1.06 (95 % CI 
0.84 to 1.33). Combined, there was a 16 % higher risk of serious adverse events in MRNA vaccine recip- 
ients: risk difference 13.2 (95 % CI —3.2 to 29.6); risk ratio 1.16 (95 % CI 0.97 to 1.39). 
Discussion: The excess risk of serious adverse events found in our study points to the need for formal 
harm-benefit analyses, particularly those that are stratified according to risk of serious COVID-19 out- 
comes. These analyses will require public release of participant level datasets. 
© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http:// 
creativecommons.org/licenses/by/4.0/). 


1. Introduction 


updated a “priority list of potential adverse events of special inter- 
est relevant to COVID-19 vaccine trials.” [1] The list comprises 


In March 2020, the Brighton Collaboration and the Coalition for 
Epidemic Preparedness Innovations partnership, Safety Platform 
for Emergency vACcines (SPEAC), created and subsequently 
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adverse events of special interest (AESIs) based on the specific vac- 
cine platform, adverse events associated with prior vaccines in 
general, theoretical associations based on animal models, and 
COVID-19 specific immunopathogenesis. | 1] The Brighton Collabo- 
ration is a global authority on the topic of vaccine safety and in 
May 2020, the World Health Organization’s Global Advisory Com- 
mittee on Vaccine Safety endorsed and recommended the report- 
ing of AESIs based on this priority list. To our knowledge, 
however, the list has not been applied to serious adverse events 
in randomized trial data. 


This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). 
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We sought to investigate the association between FDA- 
authorized mRNA COVID-19 vaccines and serious adverse events 
identified by the Brighton Collaboration, using data from the phase 
III randomized, placebo-controlled clinical trials on which autho- 
rization was based. We consider these trial data against findings 
from post-authorization observational safety data. Our study was 
not designed to evaluate the overall harm-benefit of vaccination 
programs so far. To put our safety results in context, we conducted 
a simple comparison of harms with benefits to illustrate the need 
for formal harm-benefit analyses of the vaccines that are stratified 
according to risk of serious COVID-19 outcomes. Our analysis is 
restricted to the randomized trial data, and does not consider data 
on post-authorization vaccination program impact. It does how- 
ever show the need for public release of participant level trial 
datasets. 


2. Methods 


Pfizer and Moderna each submitted the results of one phase III 
randomized trial in support of the FDA’s emergency use authoriza- 
tion of their vaccines in adults. Two reviewers (PD and RK) 
searched journal publications and trial data on the FDA’s and 
Health Canada’s websites to locate serious adverse event results 
tables for these trials. The Pfizer and Moderna trials are expected 
to follow participants for two years. Within weeks of the emer- 
gency authorization, however, the sponsors began a process of 
unblinding all participants who elected to be unblinded. In addi- 
tion, those who received placebo were offered the vaccine. These 
self-selection processes may have introduced nonrandom differ- 
ences between vaccinated and unvaccinated participants, thus ren- 
dering the post-authorization data less reliable. Therefore, to 
preserve randomization, we used the interim datasets that were 
the basis for emergency authorization in December 2020, approx- 
imately 4 months after trials commenced. 

The definition of a serious adverse event (SAE) was provided in 
each trial’s study protocol and included in the supplemental mate- 
rial of the trial’s publication. [2-4] Pfizer and Moderna used nearly 
identical definitions, consistent with regulatory expectations. An 
SAE was defined as an adverse event that results in any of the fol- 
lowing conditions: death; life-threatening at the time of the event; 
inpatient hospitalization or prolongation of existing hospitaliza- 
tion; persistent or significant disability/incapacity; a congenital 
anomaly/birth defect; medically important event, based on medi- 
cal judgment. 

In addition to journal publications, we searched the websites of 
the FDA (for advisory committee meeting materials) and Health 
Canada (for sections of the dossier submitted by sponsors to the 
regulator). [5] For the FDA website, we considered presentations 
by both the FDA and the sponsors. [6] Within each of these sources, 
we searched for SAE results tables that presented information by 
specific SAE type; we chose the most recent SAE table correspond- 
ing to the FDA’s requirement for a safety median follow-up time of 
at least 2 months after dose 2. 

For each trial, we prepared blinded SAE tables (containing SAE 
types without results data). Using these blinded SAE tables, two 
clinician reviewers (JF and JE) independently judged whether each 
SAE type was an AESI. SAE types that matched an AESI term verba- 
tim, or were an alternative diagnostic name for an AESI term, were 
included as an AESI. For all other SAE types, the reviewers indepen- 
dently judged whether that SAE type was likely to have been 
caused by a vaccine-induced AESI, based on a judgment consider- 
ing the disease course, causative mechanism, and likelihood of 
the AESI to cause the SAE type. Disagreements were resolved 
through consensus; if consensus could not be reached, a third clin- 
ician reviewer (PW) was used to create a majority opinion. For each 
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included SAE, we recorded the corresponding Brighton Collabora- 
tion AES] category and organ system. When multiple AESIs could 
potentially cause the same SAE, the reviewers selected the AESI 
that they judged to be the most likely cause based on classical clin- 
ical presentation of the AESI. 

We used an AESI list derived from the work of Brighton Collab- 
oration’s Safety Platform for Emergency vACcines (SPEAC) Project. 
This project created an AESI list which categorizes AESIs into three 
categories: those included because they are seen with COVID-19, 
those with a proven or theoretical association with vaccines in 
general, and those with proven or theoretical associations with 
specific vaccine platforms. The first version was produced in March 
2020 based on experience from China. Following the second 
update (May 2020), the WHO Global Advisory Committee on Vac- 
cine Safety (GACVS) adopted the list, and Brighton commenced a 
systematic review process “to ensure an ongoing understanding 
of the full spectrum of COVID-19 disease and modification of the 
AESI list accordingly.” [7] This resulted in three additional AESIs 
being added to the list in December 2020. The subsequent (and 
most recent fourth) update did not result in any additional AESIs 
being added to the list. [1]. 

We matched SAEs recorded in the trial against an expanded list 
of AESIs created by combining Brighton’s SPEAC COVID-19 AESI list 
with a list of 29 clinical diagnoses Brighton identified as “known to 
have been reported but not in sufficient numbers to merit inclu- 
sion on the AESI list.” [7] Sensitivity analysis was used to deter- 
mine whether use of the original versus expanded list altered our 
results. 

Risk ratios and risk differences between vaccine and placebo 
groups were calculated for the incidence of AESIs and SAEs. We 
excluded SAEs that were known efficacy outcomes (i.e. COVID- 
19), consistent with the approach Pfizer (but not Moderna) used 
in recording SAE data. The Pfizer study trial protocol states that 
COVID-19 illnesses and their sequelae consistent with the clinical 
endpoint definition were not to be reported as adverse events, 
“even though the event may meet the definition of an SAE.” [8] 
For unspecified reasons, Moderna included efficacy outcomes in 
their SAE tables, effectively reporting an all-cause SAE result. 
Because we did not have access to individual participant data, to 
account for the occasional multiple SAEs within single participants, 
we reduced the effective sample size by multiplying standard 
errors in the combined SAE analyses by the square root of the ratio 
of the number of SAEs to the number of patients with an SAE. This 
adjustment increased standard errors by 10 % (Pfizer) and 18 % 
(Moderna), thus expanding the interval estimates. We estimated 
combined risk ratios and risk differences for the two mRNA vacci- 
nes by averaging over the risks using logistic regression models 
which included indicators for trial and treatment group. 

We used a simple harm-benefit framework to place our results 
in context, comparing risks of excess serious AESIs against reduc- 
tions in COVID-19 hospitalization. 


3. Results 


Serious adverse event tables were located for each of the vac- 
cine trials submitted for EUA in adults (age 16 + for Pfizer, 
18 + for Moderna) in the United States: Pfizer-BioNTech COVID- 
19 vaccine BNT162b2 (NCT04368728) [2,9,10] and Moderna 
COVID-19 vaccine mRNA-1273 (NCT04470427). [3,11,12] 
(Table 1). 


3.1. Reporting windows and serious adverse events 


Moderna reported SAEs from dose 1 whereas Pfizer limited 
reporting from dose 1 to 1 month after dose 2. Both studies 
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Table 1 
Data sources for phase III trials. 
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Trial Data cutoff date Journal FDA sources Health Canada sources 
articles 
Pfizer trial in ages 16 and above 14 Nov 2020 (supported Aggregate Table 23 in sponsor Table 55 in sponsor document C4591001 Final Analysis 
(NCT04368728) Dec 2020 EUA) data only briefing document Interim Report Body 
Moderna trial in ages 18 and 25 Nov 2020 (supported Table S11 in Table 27 in sponsor Table 14.3.1.13.3 in sponsor document mRNA-1273-P301 
above (NCT04470427) Dec 2020 EUA) publication briefing document Unblinded Safety Tables Batch 1 (DS2) 


Note: bolded font indicates dataset chosen for analysis; EUA = Emergency Use Authorization. 


reported all data at the time of data cutoff (14 Nov 2020 for Pfizer, 
25 Nov 2020 for Moderna). 17 SAEs that were efficacy endpoints 
were removed from the Moderna trial (16 “COVID-19” SAEs and 
1 “COVID-19 pneumonia” SAE). One such efficacy endpoint meet- 
ing the definition of a SAE was removed from the Pfizer trial 
(“SARS-CoV-2 test positive” SAE). 

The Pfizer trial exhibited a 36 % higher risk of serious adverse 
events in vaccinated participants in comparison to placebo recipi- 
ents: 67.5 per 10,000 versus 49.5 per 10,000; risk difference 18.0 
per 10,000 vaccinated participants (95 % compatibility’ interval 
1.2 to 34.9); risk ratio 1.36 (95 % CI 1.02 to 1.83). The Moderna trial 
exhibited a 6 % higher risk of SAEs in vaccinated individuals com- 
pared to those receiving placebo: 136 per 10,000 versus 129 per 
10,000; risk difference 7.1 per 10,000 (95 % CI -23.2 to 37.4); risk 
ratio 1.06 (95 % CI 0.84 to 1.33). Combined, there was a 16 % higher 
risk of SAEs in mRNA vaccine recipients than placebo recipients: 98 
per 10,000 versus 85 per 10,000; risk difference 13.2 (95 % CI —3.2 to 
29.6); risk ratio 1.16 (95 % CI 0.97 to 1.39). (Table 2). 


3.2. Serious adverse events of special interest 


Regarding whether each SAE type was included on the SPEAC 
derived AESI list, agreement between the two independent clini- 
cian reviewers was 86 % (281/325); 40 of the 44 disagreements 
were resolved through consensus, and only four disagreements 
necessitated a third clinician reviewer. Supplemental Table 1 
includes a full list of included and excluded SAEs across both trials. 

In the Pfizer trial, 52 serious AESI (27.7 per 10,000) were 
reported in the vaccine group and 33 (17.6 per 10,000) in the pla- 
cebo group. This difference corresponds to a 57 % higher risk of 
serious AESI (RR 1.57 95 % CI 0.98 to 2.54) and a risk difference 
of 10.1 serious AESI per 10,000 vaccinated participants (95 % CI 
—0.4 to 20.6). In the Moderna trial, 87 serious AESI (57.3 per 
10,000) were reported in the vaccine group and 64 (42.2 per 
10,000) in the placebo group. This difference corresponds to a 
36 % higher risk of serious AESI (RR 1.36 95 % CI 0.93 to 1.99) 
and a risk difference of 15.1 serious AESI per 10,000 vaccinated 
participants (95 % CI —3.6 to 33.8). Combining the trials, there 
was a 43 % higher risk of serious AESI (RR 1.43; 95 % CI 1.07 to 
1.92) and a risk difference of 12.5 serious AESI per 10,000 vacci- 
nated participants (95 % CI 2.1 to 22.9). (Table 2). 

Of the 236 serious AESIs occurring across the Pfizer and Mod- 
erna trials, 97 % (230/236) were adverse event types included as 
AESIs because they are seen with COVID-19. In both Pfizer and 
Moderna trials, the largest excess risk occurred amongst the 
Brighton category of coagulation disorders. Cardiac disorders have 
been of central concern for mRNA vaccines; in the Pfizer trial more 
cardiovascular AESIs occurred in the vaccine group than in the pla- 
cebo group, but in the Moderna trial the groups differed by only 1 
case. (Tables 3 and 4). 


1 A compatibility interval is identical to a confidence interval, but relabeled to 
emphasize that it is not a Bayesian posterior interval (as is improperly suggested by 
the “confidence” label).!*"4. 
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3.3. Sensitivity analysis 


As a Sensitivity analysis, we restricted the serious AESI analysis 
to those AESIs listed in SPEAC’s COVID-19 AESI list (i.e. separating 
out Brighton’s list of 29 clinical diagnoses “known to have been 
reported but not in sufficient numbers to merit inclusion on the 
AESI list.”) This reduced the total number of AESIs across the two 
trials by 48 (35 vaccine group, 13 placebo group). There was still 
a higher risk of serious AESI when limited to the SPEAC COVID- 
19 AESI list, but the magnitude of the excess (in both relative 
and absolute terms) was smaller than when using the larger AESI 
list. (Supplemental Table 2). 


3.4, Harm-benefit considerations 


In the Moderna trial, the excess risk of serious AESIs (15.1 per 
10,000 participants) was higher than the risk reduction for 
COVID-19 hospitalization relative to the placebo group (6.4 per 
10,000 participants). [3] In the Pfizer trial, the excess risk of serious 
AESIs (10.1 per 10,000) was higher than the risk reduction for 
COVID-19 hospitalization relative to the placebo group (2.3 per 
10,000 participants). 


4. Comparison with FDA reviews 


In their review of SAEs supporting the authorization of the Pfi- 
zer and Moderna vaccines, the FDA concluded that SAEs were, for 
Pfizer, “balanced between treatment groups,” [15] and for Mod- 
erna, were “without meaningful imbalances between study arms.” 
[16] In contrast to the FDA analysis, we found an excess risk of 
SAEs in the Pfizer trial. Our analysis of Moderna was compatible 
with FDA’s analysis, finding no meaningful SAE imbalance between 
groups. 

The difference in findings for the Pfizer trial, between our SAE 
analysis and the FDA’s, may in part be explained by the fact that 
the FDA analyzed the total number of participants experiencing 
any SAE, whereas our analysis was based on the total number of 
SAE events. Given that approximately twice as many individuals 
in the vaccine group than in the placebo group experienced multi- 
ple SAEs (there were 24 more events than participants in the vac- 
cine group, compared to 13 in the placebo group), FDA’s analysis of 
only the incidence of participants experiencing any SAE would not 
reflect the observed excess of multiple SAEs in the vaccine group. 

A more important factor, however, may be that FDA’s review of 
non-fatal SAEs used a different analysis population with different 
follow-up windows. The FDA reported 126 of 21,621 (0.6 %) of vac- 
cinated participants experienced at least one SAE at data cutoff 
compared to 111 of 21,631 (0.5 %) of placebo participants. In con- 
trast, our analysis found 127 SAEs among 18,801 vaccine recipients 
versus 93 SAEs among 18,785 placebo recipients. [15] While sum- 
mary results for the population we analyzed was provided in a 
table, FDA did not report an analysis of them. The substantially lar- 
ger denominators in FDA’s analysis (5,666 more participants) 
reflect the fact that their analysis included all individuals receiving 
at least one dose (minus 196 HIV-positive participants), irrespec- 
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Table 2 
Serious adverse events. 
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Total events (events per 10,000 Risk difference Risk ratio 
participants)* per 10,000 participants (95 % CI): 
Trial Vaccine Placebo (95 % CI)° 
Serious adverse events 
Pfizer? 127 (67.5) 93 (49.5) 18.0 (1.2 to 34.9) 1.36 (1.02 to 1.83) 
Moderna‘ 206 (135.7) 195 (128.6) 7.1 (-23.2 to 37.4) 1.06 (0.84 to 1.33) 
Combined! 333 (98.0) 288 (84.8) 13.2 (-3.2 to 29.6) 1.16 (0.97 to 1.39) 
Serious adverse events of special interest 
Pfizer 52 (27.7) 33 (17.6) 10.1 (-0.4 to 20.6) 1.57 (0.98 to 2.54) 
Moderna 87 (57.3) 64 (42.2) 15.1 (-3.6 to 33.8) 1.36 (0.93 to 1.99) 
Combined’ 139 (40.9) 97 (28.6) 12.5 (2.1 to 22.9) 1.43 (1.07 to 1.92) 


* Denominators for Pfizer were 18,801 in the vaccine group and 18,785 in the placebo group, and for Moderna were 15,185 in the vaccine group and 15,166 in the placebo 


group. 


> Pfizer excluded efficacy outcomes from its SAE table (COVID-19 illnesses and their sequelae meeting the definition of an SAE). However, at least one SAE appears to have 
been inadvertently included, which we removed from our calculations (“SARS-CoV-2 test positive”: 0 vaccine group; 1 placebo group). 

© Moderna included efficacy outcomes in its SAE table (COVID-19 illnesses and their sequelae meeting the definition of an SAE). We removed efficacy SAEs outcomes that 
could be identified: “COVID-19” and “COVID-19 pneumonia.” Lacking access to participant level data, SAEs that were sequelae of serious COVID-19 could not be identified and 


therefore remain included in this analysis. 


d “All SAEs” for Moderna was calculated using the “Number of serious AEs” row in Moderna’s submission to FDA.!'. 
© Standard errors used to estimate 95% CIs were inflated by the factor \/[#SAE]/[#patients with SAE] to account for multiple SAE within patients. 
f The combined risk differences and risk ratios were computed from the fitted logistic regression models and so may not exactly equal comparisons computed from the first 


two columns. 


Table 3 

Serious AESIs, Pfizer trial. 
Brighton category Vaccine Placebo Vaccine events per 10,000 Placebo events per 10,000 Difference in events per 10,000 Risk ratio 
Association with immunization in general 
Anaphylaxis 1 1 0.5 0.5 0.0 1.00 
Association with specific vaccine platform(s) 
Encephalitis/encephalomyelitis 0 2 0.0 1.1 -1.1 0.00 
Seen with COVID-19 
Acute kidney injury 2 0 1.1 0.0 1.1 N/A 
Acute liver injury 0 1 0.0 0.5 —0.5 0.00 
Acute respiratory distress syndrome 2 1 1.1 0.5 0.5 2.00 
Coagulation disorder 16 10 8.5 5.3 3.2 1.60 
Myocarditis/pericarditis 2 1 1.1 0.5 0.5 2.00 
Other forms of acute cardiac injury 16 12 8.5 6.4 2.1 1.33 
Subtotal 39 28 20.7 14.9 5.8 1.39 
Brighton list of 29 clinical diagnoses seen with COVID-19 
Abscess 4 1 2.1 0.5 1.6 4.00 
Cholecystitis 4 2 2.1 1.1 1.1 2.00 
Colitis/Enteritis 1 1 0.5 0.5 0.0 1.00 
Diarrhea 1 0 0.5 0.0 0.5 N/A 
Hyperglycemia 1 1 0.5 0.5 0.0 1.00 
Pancreatitis 1 0 0.5 0.0 0.5 N/A 
Psychosis 1 (0) 0.5 0.0 0.5 N/A 
Subtotal 13 5 6.9 2.7 4.3 2.60 
Total 52 33 27.7 17.6 10.1 1.57 


tive of the duration of post-injection follow-up time. In contrast, 
our analysis was based on the study population with median 
follow-up > 2 months after dose 2 (minus 120 HIV-positive partic- 
ipants), of which 98.1 % had received both doses. [2,17] The FDA’s 
analysis of SAEs thus included thousands of additional participants 
with very little follow-up, of which the large majority had only 
received 1 dose. 


4.1. Comparison with post-authorization studies 


Although the randomized trials offer high level evidence for 
evaluating causal effects, the sparsity of their data necessitates that 
harm-benefit analyses also consider observational studies. Since 
their emergency authorization in December 2020, hundreds of mil- 
lions of doses of Pfizer and Moderna COVID-19 vaccines have been 
administered and post-authorization observational data offer a 
complementary opportunity to study AESIs. Post-authorization 
observational safety studies include cohort studies (which make 
use of medical claims or electronic health records) and dispropor- 
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tionality analyses (which use spontaneous adverse event reporting 
systems). In July 2021, the FDA reported detecting four potential 
adverse events of interest: pulmonary embolism, acute myocardial 
infarction, immune thrombocytopenia, and disseminated intravas- 
cular coagulation following Pfizer's vaccine based on medical 
claims data in older Americans. [18] Three of these four serious 
adverse event types would be categorized as coagulation disorders, 
which is the Brighton AESI category that exhibited the largest 
excess risk in the vaccine group in both the Pfizer and Moderna tri- 
als. FDA stated it would further investigate the findings but at the 
time of our writing has not issued an update. Similarly, 
spontaneous-reporting systems have registered serious adverse 
reactions including anaphylaxis (all COVID-19 vaccines), thrombo- 
cytopenia syndrome among premenopausal females (Janssen vac- 
cine), and myocarditis and pericarditis among younger males 
(Pfizer and Moderna vaccines). [19,20]. 

Using data from three postmarketing safety databases for vacci- 
nes (VAERS, EudraVigilance, and VigiBase), disproportionality stud- 
ies have reported excess risks for many of the same SAE types as in 
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Table 4 

Serious AESIs, Moderna trial. 
Brighton category Vaccine Placebo Vaccine events per 10,000 Placebo events per 10,000 Difference in events per 10,000 Risk ratio 
Association with specific vaccine platform(s) 
Bell's Palsy 1 0 0.7 0.0 0.7 N/A 
Encephalitis/encephalomyelitis 1 0 0.7 0.0 0.7 N/A 
Seen with COVID-19 
Acute kidney injury 1 3 0.7 2.0 -1.3 0.33 
Acute liver injury 1 0 0.7 0.0 0.7 N/A 
Acute respiratory distress syndrome 7 4 4.6 2.6 2.0 1.75 
Angioedema 0 2 0.0 13 -1.3 0.00 
Coagulation disorder 20 13 13.2 8.6 4.6 1.54 
Generalized Convulsions 2 (0) 1.3 0.0 13 N/A 
Myelitis 0 1 0.0 0.7 —0.7 0.00 
Myocarditis/pericarditis 4 5 2.6 3.3 —0.7 0.80 
Other forms of acute cardiac injury 26 26 17.1 17.1 0.0 1.00 
Other rash 1 1 0.7 0.7 0.0 1.00 
Rhabdomyolysis 0 1 0.0 0.7 —0.7 0.00 
Single Organ Cutaneous Vasculitis 1 0 0.7 0.0 0.7 N/A 
Subtotal 65 56 42.8 36.9 5.9 1.16 
Brighton list of 29 clinical diagnoses seen with COVID-19 
Abscess 1 (0) 0.7 0.0 0.7 N/A 
Arthritis 3 1 2.0 0.7 1.3 3.00 
Cholecystitis 4 (0) 2.6 0.0 2.6 N/A 
Colitis/Enteritis 6 3 4.0 2.0 2.0 2.00 
Diarrhea 2 1 1.3 0.7 0.7 2.00 
Hyperglycemia 1 0 0.7 0.0 0.7 N/A 
Hyponatremia 1 1 0.7 0.7 0.0 1.00 
Pancreatitis 2 0 1.3 0.0 13 N/A 
Pneumothorax 0 1 0.0 0.7 —0.7 0.00 
Psychosis 1 1 0.7 0.7 0.0 1.00 
Thyroiditis 1 0 0.7 0.0 0.7 N/A 
Subtotal 22 8 14.5 5.3 9.2 2.75 
Total 87 64 57.3 42.2 15.1 1.36 


the present study. [21-23] For example, a study using VAERS and 
EudraVigilance comparing the disproportionality of adverse event 
reports between the influenza vaccine versus the mRNA COVID- 
19 vaccines reported excess risks for the following Brighton AESIs: 
cardiovascular events, coagulation events, hemorrhages, gastroin- 
testinal events, and thromboses. [22] While CDC published a proto- 
col[24] in early 2021 for using proportional reporting ratios for 
signal detection in the VAERS database, results from the study have 
not yet been reported. [25] Among self-controlled case series, one 
reported a rate ratio of 1.38 (95 % CI 1.12-1.71) for hemorrhagic 
stroke following Pfizer vaccine, [26] another reported 0.97 (95 % 
CI 0.81-1.15), [27] while a cohort study[28] reported 0.84 (95 % 
CI 0.54-1.27). 


5. Discussion 


Using a prespecified list of AESI identified by the Brighton Col- 
laboration, higher risk of serious AESI was observed in the mRNA 
COVID-19 vaccine group relative to placebo in both the Pfizer 
and Moderna adult phase III trials, with 10.1 (Pfizer) and 15.1 
(Moderna) additional events for every 10,000 individuals vacci- 
nated. Combined, there was a risk difference of 12.5 serious AESIs 
per 10,000 individuals vaccinated (95 % CI 2.1 to 22.9). These 
results raise concerns that mRNA vaccines are associated with 
more harm than initially estimated at the time of emergency 
authorization. In addition, our analysis identified a 36 % higher risk 
of serious adverse events in vaccinated participants in the Pfizer 
trial: 18.0 additional SAEs per 10,000 vaccinated (95 % CI 1.2 to 
34.9). Consistent with the FDA evaluation, our analysis found no 
clear difference in SAEs between groups in the Moderna trial. 

Results between the Pfizer and Moderna trials were similar for 
the AESI analysis but exhibited substantial variation in the SAE 
analysis. Caution is needed in interpreting this variation as it 
may be substantially explained by differences in SAE recording 
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practices in the trials rather than differences in actual vaccine 
harm profiles. For reasons that are not documented in the trial pro- 
tocol, Moderna included efficacy outcomes in its SAE tabulations, 
while Pfizer excluded them. As a result, Moderna’s SAE table did 
not present a traditional SAE analysis but rather an all-cause SAE 
analysis. The FDA analysis of the Moderna trial presented an all- 
cause SAE analysis, which estimates total vaccine effects on SAEs, 
including effects transmitted via effects on COVID-19. It did not 
however present a traditional SAE analysis with efficacy endpoints 
removed, which attempts to estimate only the direct effects on 
SAEs. While our analysis attempted to perform a traditional SAE 
analysis by excluding efficacy SAEs (serious COVID-19 and its 
sequelae), our effort was hindered because we did not have access 
to patient level data. Easily recognizable efficacy SAEs (“COVID- 
19”, “COVID-19 pneumonia,” and “SARS-CoV-2 test positive”) 
could be removed, but many participants who experienced a 
COVID-19 SAE likely experienced multiple other SAEs (e.g. pneu- 
monia, hypoxia, and thrombotic events) which could not be iden- 
tified and therefore remain included in our analysis. Of 17 total 
efficacy SAEs (16 “COVID-19” and 1 “COVID-19 pneumonia”) 
removed from our analysis of the Moderna trial, 16 were in the pla- 
cebo arm. As a consequence, the background SAE risk (risk in 
absence of COVID-19) would be overestimated by the Moderna 
placebo group, resulting in underestimation of the actual risk of 
SAEs and AESIs attributable to the vaccine in the Moderna compar- 
isons as well as in the combined analysis. Access to patient-level 
data would allow adjustments for this problem. 

Rational policy formation should consider potential harms 
alongside potential benefits. [29] To illustrate this need in the pre- 
sent context, we conducted a simple harm-benefit comparison 
using the trial data comparing excess risk of serious AESI against 
reductions in COVID-19 hospitalization. We found excess risk of 
serious AESIs to exceed the reduction in COVID-19 hospitalizations 
in both Pfizer and Moderna trials. 
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This analysis has the limitations inherent in most harm-benefit 
comparisons. First, benefits and harms are rarely exact equivalents, 
and there can be great variability in the degree of severity within 
both benefit and harm endpoints. For example, intubation and 
short hospital stay are not equivalent but both are counted in 
“hospitalization”; similarly, serious diarrhea and serious stroke 
are not equivalent but both are counted in “SAE.” Second, individ- 
uals value different endpoints differently. Third, without individual 
participant data, we could only compare the number of individuals 
hospitalized for COVID-19 against the number of serious AESI 
events, not the number of participants experiencing any serious 
AESI. Some individuals experienced multiple SAEs whereas hospi- 
talized COVID-19 participants were likely only hospitalized once, 
biasing the analysis towards exhibiting net harm. To gauge the 
extent of this bias, we considered that there were 20 % (Pfizer) 
and 34 % (Moderna) more SAEs than participants experiencing 
any SAE. As a rough sensitivity calculation, if we divide the Pfizer 
excess serious AESI risk of 10.1 by 1.20 it becomes 8.4 compared 
to a COVID-19 hospitalization risk reduction of 2.3; if we divide 
the Moderna excess serious AESI risk of 15.1 by 1.34 it becomes 
11.3 compared to a COVID-19 hospitalization risk reduction of 6.4. 

Harm-benefit ratios will be different for populations at different 
risk for serious COVID-19 and observation periods that differ from 
those studied in the trials. Presumably, larger reductions in COVID- 
19 hospitalizations would have been recorded if trial follow-up 
were longer, more SARS-CoV-2 was circulating, or if participants 
had been at higher risk of serious COVID-19 outcomes, shifting 
harm-benefit ratios toward benefit. Conversely, harm-benefit 
ratios would presumably shift towards harm for those with lower 
risk of serious COVID-19 outcomes--such as those with natural 
immunity, younger age or no comorbidities. Similarly, waning vac- 
cine effectiveness, decreased viral virulence, and increasing degree 
of immune escape from vaccines might further shift the harm- 
benefit ratio toward harm. Large, randomized trials in contempo- 
rary populations could robustly answer these questions. Absent 
definitive trials, however, synthesis of multiple lines of evidence 
will be essential. [30,48,49]. 

Adverse events detected in the post-marketing period have led 
to the withdrawal of several vaccines. An example is intussuscep- 
tion following one brand of rotavirus vaccine: around 1 million 
children were vaccinated before identification of intussusception, 
which occurred in around 1 per 10,000 vaccinees. [31] Despite 
the unprecedented scale of COVID-19 vaccine administration, the 
AESI types identified in our study may still be challenging to detect 
with observational methods. Most observational analyses are 
based on comparing the risks of adverse events “observed” against 
a background (or “‘expected”) risk, which inevitably display great 
variation, by database, age group, and sex. [32] If the actual risk 
ratio for the effect was 1.4 (the risk ratio of the combined AESI 
analysis), it could be quite difficult to unambiguously replicate it 
with observational data given concerns about systematic as well 
as random errors. [33-35]. 

In addition, disproportionality analyses following COVID-19 
vaccination also have limitations, particularly with respect to the 
type of adverse events seen in our study. The majority of SAEs that 
contributed to our results are relatively common events, such as 
ischemic stroke, acute coronary syndrome, and brain hemorrhage. 
This complicates signal detection because clinical suspicion of an 
adverse vaccine reaction following an event commonly seen in 
clinical practice will be lower than for SAEs like myocarditis.[50] 
For this reason, clinical suspicion leading to the filing of an individ- 
ual case safety report--may be far less common in the post- 
authorization setting than in the trials. At the same time, height- 
ened awareness about COVID-19 vaccine SAEs can result in under 
and overreporting. Public health messages assuring vaccine safety 
may lower clinical suspicion of potential causal relationships, 
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whereas messages about potential harms can conversely stimulate 
reports that otherwise may not have been made. These factors can 
lead to bias both directions, further complicating interpretation. In 
contrast to these problems, in the randomized trials used in this 
analysis, all SAEs were to be recorded, irrespective of clinical judg- 
ment regarding potential causality. 

Although our analysis is secondary, reanalyses of clinical trial 
data have led to the detection of adverse events well after the mar- 
ket entry of major drugs such as rofecoxib and rosiglitazone. 
[36,37] Our analysis has an advantage over postmarketing observa- 
tional studies in that the data are from blinded, placebo-controlled 
randomized trials vetted by the FDA, which were matched against 
a list of adverse events created before the availability of the 
clinical-trial results and designed for use in COVID-19 vaccine 
trials. 

Our study has several important limitations. First, Pfizer’s trial 
did not report SAEs occurring past 1 month after dose 2. This 
reporting threshold may have led to an undercounting of serious 
AESIs in the Pfizer trial. Second, for both studies, the limited follow 
up time prevented an analysis of harm-benefit over a longer per- 
iod. Third, all SAEs in our analysis met the regulatory definition 
of a serious adverse event, but many adverse event types which 
a patient may themselves judge as serious may not meet this reg- 
ulatory threshold. Fourth, decisions about which SAEs to include or 
exclude as AESIs requires subjective, clinical judgements in the 
absence of detailed clinical information about the actual SAEs. 
We encourage third party replication of our study, with access to 
complete SAE case narratives, to determine the degree to which 
these decisions affected our findings. For additional sensitivity 
analyses, such replication studies could also make use of other AESI 
lists, such as those prepared by FDA, [38-41] CDC, [24], Pfizer, [42], 
or a de novo AESI list derived from a list of COVID-19 complications 
understood to be induced via SARS-CoV-2’s spike protein. [43,44]. 

A fifth important limitation is our lack of access to individual 
participant data, which forced us to use a conservative adjustment 
to the standard errors. The 95 % ClIs{ 13,14] calculated are therefore 
only approximate because we do not know which patients had 
multiple events. Finally, as described above, in the Moderna anal- 
ysis, the SAEs that were sequelae of serious COVID-19 could not 
be identified and therefore remain included in our calculations. 
Because the vaccines prevent SAEs from COVID-19 while adding 
SAE risks of their own, this inclusion makes it impossible to sepa- 
rately estimate SAEs due to the vaccine from SAEs due to COVID-19 
in the available Moderna data, as must be done to extrapolate 
harm-benefit to other populations. These study limitations all stem 
from the fact that the raw data from COVID-19 vaccine clinical tri- 
als are not publicly available. [45,46]. 

We emphasize that our investigation is preliminary, to point to 
the need for more involved analysis. The risks of serious AESIs in 
the trials represent only group averages. SAEs are unlikely to be 
distributed equally across the demographic subgroups enrolled in 
the trial, and the risks may be substantially less in some groups 
compared to others. Thus, knowing the actual demographics of 
those who experienced an increase in serious AESI in the vaccine 
group is necessary for a proper harm-benefit analysis. In addition, 
clinical studies are needed to see if particular SAEs can be linked to 
particular vaccine ingredients as opposed to unavoidable conse- 
quences of exposure to spike protein, as future vaccines could then 
be modified accordingly or sensitivities can be tested for in 
advance. In parallel, a systematic review and meta-analysis using 
individual participant data should be undertaken to address ques- 
tions of harm-benefit in various demographic subgroups, particu- 
larly in those at low risk of serious complications from COVID- 
19. Finally, there is a pressing need for comparison of SAEs and 
harm-benefit for different vaccine types; some initial work has 
already begun in this direction. [47]. 
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Full transparency of the COVID-19 vaccine clinical trial data is 
needed to properly evaluate these questions. Unfortunately, as 
we approach 2 years after release of COVID-19 vaccines, partici- 
pant level data remain inaccessible. [45,46]. 
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